Skip to content

Conversation

Pouyanpi
Copy link
Collaborator

@Pouyanpi Pouyanpi commented Jul 21, 2025

This pull request introduces a new integration for Guardrails AI validators into the NeMo Guardrails framework. It adds support for configuring, registering, and invoking Guardrails AI validators on both input and output text, along with robust error handling and dynamic validator loading. The changes span configuration, core logic, error handling, flow definitions, and registry management.

The most important changes are:

Guardrails AI Validator Integration & Core Logic:

  • Introduced a new GuardrailsAIRailConfig and GuardrailsAIValidatorConfig in nemoguardrails/rails/llm/config.py to allow users to specify and configure multiple Guardrails AI validators, including parameters and metadata. Added a method for retrieving validator configs by name. [1] [2]
  • Implemented dynamic loading and invocation of Guardrails AI validators in nemoguardrails/library/guardrails_ai/actions.py, including caching, error handling, and mapping of validation results to the NeMo Guardrails format. Added two main actions: validate_guardrails_ai_input and validate_guardrails_ai_output.

Configuration & Flow Integration:

  • Added an example configuration file examples/configs/guardrails_ai/config.yml demonstrating how to specify models and set up input/output validation flows using Guardrails AI validators.
  • Defined new Colang flows (flows.co and flows.v1.co) to check input and output text with the specified validators, handling failures by either raising exceptions or refusing to respond. [1] [2]

Validator Registry & Error Handling:

  • Added a registry in nemoguardrails/library/guardrails_ai/registry.py for mapping validator names to their module/class/hub paths, and logic to fetch validator info from Guardrails Hub if not found in the local registry.
  • Introduced custom error classes in nemoguardrails/library/guardrails_ai/errors.py for handling configuration and validation errors, with graceful fallback if Guardrails is not installed.

Miscellaneous:

  • Added license headers and initial module files for the new Guardrails AI integration. (__init__.py, actions, errors, registry) [1] [2] [3] [4]

These changes collectively enable seamless integration of Guardrails AI validators into the NeMo Guardrails ecosystem, with flexible configuration, dynamic loading, and robust flow control.

@Pouyanpi Pouyanpi force-pushed the feat/guardrailsai-integration branch 2 times, most recently from ab11205 to 71b4a0b Compare July 21, 2025 15:21
@Pouyanpi Pouyanpi force-pushed the feat/guardrailsai-integration branch from 71b4a0b to a12dc8a Compare July 31, 2025 12:12
@Pouyanpi Pouyanpi changed the base branch from releaes/v0.15.0 to develop July 31, 2025 12:31
@Pouyanpi Pouyanpi changed the title feat: community integration feat: add GuardrailsAI community integration Jul 31, 2025
@codecov-commenter
Copy link

codecov-commenter commented Jul 31, 2025

Codecov Report

❌ Patch coverage is 65.18987% with 55 lines in your changes missing coverage. Please review.
✅ Project coverage is 71.62%. Comparing base (9a1d178) to head (974a472).

Files with missing lines Patch % Lines
nemoguardrails/library/guardrails_ai/actions.py 60.74% 42 Missing ⚠️
nemoguardrails/library/guardrails_ai/registry.py 60.71% 11 Missing ⚠️
nemoguardrails/library/guardrails_ai/errors.py 90.90% 1 Missing ⚠️
nemoguardrails/rails/llm/config.py 91.66% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #1298      +/-   ##
===========================================
- Coverage    71.68%   71.62%   -0.07%     
===========================================
  Files          168      171       +3     
  Lines        16862    17020     +158     
===========================================
+ Hits         12088    12191     +103     
- Misses        4774     4829      +55     
Flag Coverage Δ
python 71.62% <65.18%> (-0.07%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
nemoguardrails/library/guardrails_ai/errors.py 90.90% <90.90%> (ø)
nemoguardrails/rails/llm/config.py 90.75% <91.66%> (+0.01%) ⬆️
nemoguardrails/library/guardrails_ai/registry.py 60.71% <60.71%> (ø)
nemoguardrails/library/guardrails_ai/actions.py 60.74% <60.74%> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@Pouyanpi Pouyanpi added this to the v0.16.0 milestone Aug 1, 2025
@Pouyanpi
Copy link
Collaborator Author

Pouyanpi commented Aug 7, 2025

@zsimjee please have a look.

If you are OK with this approach we can follow two options: 1) we continue working on it on this branch and eventually merge it and close #810 2) we cleanup #810 and target this PR to that branch.

@Pouyanpi Pouyanpi modified the milestones: v0.16.0, v0.15.0 Aug 7, 2025
@Pouyanpi Pouyanpi marked this pull request as ready for review August 18, 2025 11:14
@zsimjee
Copy link

zsimjee commented Aug 20, 2025

Hi Pouyan! This looks great, happy to work on it here. I think I just need to pull down/test through

@zsimjee
Copy link

zsimjee commented Aug 26, 2025

tested this out, it looks great!

@Pouyanpi Pouyanpi force-pushed the feat/guardrailsai-integration branch from ef55f83 to c03679a Compare August 27, 2025 10:05
@Pouyanpi Pouyanpi merged commit 927a307 into develop Aug 27, 2025
17 checks passed
@Pouyanpi Pouyanpi deleted the feat/guardrailsai-integration branch August 27, 2025 10:24
@Pouyanpi
Copy link
Collaborator Author

Thank you @zsimjee, merged. It will be part of 0.16.0 release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants